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ABSTRACT 

This bulletin serves as a guide to those persons 
interested in conducting or implementing educational research. 
Research is described as being "controlled inquiry", either in the 
form of an experiment or a survey. Descriptions are given of the 
procedures and complications associated with planning and initiating 
experimental research. A suggested routine would involve: (1) 

identification of the variables, (2) control of the relevant 
variables, (3) selection of an appropriate measuring instrument, (4) 
verification of the sample appropriateness, (5) a pilot study (if 
possible) , and (6) dissemination of the findings. (RS) 
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RESEARCH IN SCHOOLS 



WHAT IS Research is controlled inquiry. 

RESEARCH? 

In these bulletins, we discuss research on the elementary 

school mathematics curriculum and research on the teaching 
and learning of mathematics. The vast majority of this research is product- 
oriented; there is, however, other research which is theory-oriented. The 
task of building a theory of the learning of mathematics concepts still lies 
before us, as Begle (1968) and Glennon (1966) noted. 

Many of the studies we have cited have involved either experimental or survey 
research. By experimental we mean research in which the investigator has 
’’manipulated*' one or more specified variables, such as two methods of teaching, 
to measure their effect on another variable, such as achievement or attitude, 
thus testing a carefully formulated hypothesis or hypotheses. The variables 
which are manipulated are termed ''independent ,** while those affected and mea- 
sured are "dependent" variables. Experimental research is very difficult to 
conduct, because of the need to control the independent variable(s) and many 
other variables — which must be controlled since we want to interpret the re- 
sults and generalize beyond the sample in the study. By survey we mean re- 
search which attempts to ascertain the characteristics of a population by 
studying a sample which answers a questionnaire or interview or test. 

The material included in this bulletin is a product of the Interpretive Study 
of Research and Development in Elementary School Mathematics (Grant No. OEG— 
0-9-480586-1352(010), sponsored by the Research Utilization Branch, Bureau of 
Research, U.S. Office of Education. 

The bulletin was prepared by MARILYN N. SUYDAM, The Pennsylvania State 
University, Project Director, and vT. FRED WEAVER, The University of Wisconsin- 
Madison, Project Consultant. Art by Ed Saffell. 
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As we continue to discuss "research" in this bulletin, the focus is on experi 
mental research. You should rdcognize, however, that certain of the things 
discussed are also applicable to other types of research. We should caution 
that, despite this focus on experimental studies, we are not thus implicitly 
stating^^that such investigations are the only ones which qualify as "true re- 
search." Other types of studies also contribute to the improvement of mathe- 
matics education. 



Research is not independent of instruction. 

Jt is derived from and is applied to instruction. 
Actually, every teacher does a type of "action 
research every day — whenever new ideas are tried 
out. You're constantly trying to find the 
methods and materials and procedures which will 
work best for you. You’re assessing what pupils 
have learned, and using what you find out as you 
plan what to do next. You're concerned with what 
will help you teach better, or help your pupils 
learn better. You’ve been using evaluation , and 
for some purposes — such as curriculum development 
— evaluation is vital. 




For other purposes, however, research is essential. Research involves more 
precise controls. In experimental research, w§ are attempting to secure infor- 
mation which can be generalized to many other teachers and to many different 
situations. In survey research, we also maintain greater controls than in 
usual classroom testing — we want a more precise measurement of the status or 
level of learning. 



Research can provide a foundation on which to make curricular 
decisions and decisions about how to teach. Nothing has ever 
been proven by educational research — but it has provided 
guidelines to aid us in making decisions. It should be 
noted, however, that not all problems are amenable to research— some decisions 
must be made on the basis of your philosophy. For instance, research can pro- 
vide an answer to we teach logic to fourth graders?" but it cannot pro- 

vide an answer to "Should we teach logic to fourth graders?" 

Research has a valid role to play in assessing and improving the quality of 
instruction. In fact, merely being involved in research helps us to achieve 
this latter goal. As Pikaart and Berryman (1965) note, "Participating in re- 
search and contributing significant ideas was in itself motivating, and it 
contributed to self-esteem." 

Local school systems may at times need to engage in their own research for 
other reasons. For instance, generalized findings may not be applicable when 

unique characteristics of the system are considered (e.g., ability level of 
the pupils) . 
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HOW? First of all, select a question which is important to answer. 

Then design the study ; lay out an overall plan, delimiting 
the problem to make it researchable. This may be a long- 
term plan, but don't try to investigate everything at once: 
order your priorities logically. 

You will need to identify and define or describe (1) the independen t variable 
or variables and (2) the dependent variable or variables. You must also 
identify and control other relevant variables . Suydam (1967) reported that 
control of variables was one of the two most poorly handled facets of mathe- 
matics research studies (sampling was the other one). As Johnson (1966) noted, 
certain assumptions are made regarding what variables may affect the situation. 
During an experiment, the groups involved should have common experiences except 
for the treatment (independent) variables. Then significant differences at the 
end of the experiment can be attributed to the treatment. Johnson presents an 
example of an experiment in which many factors are controlled; Wilson (1967) 
and Worthen (1968) provide other excellent examples of research in which vari- 
ables are well-controlled. 

Some pupil variables may be controlled in one of several ways (Riedesel and 
Sparks, 1968; Kerlinger, 1964) : (1) eliminate the variable as a variable, by 

studying only a specified subset of the sample; (2) use the statistical pro- 
cedure of analysis of covariance (but be careful not to "wash out" true dif- 
ferences, as may happen when you apply covariance to a factor of concern); (3) 
incorporate the factor as another independent variable; (4) match pupil for 
pupil (this may be difficult, depending on the number of factors on which 
pupils should be matched); (5) equate on the basis of group means. Or, you 
can use randomization, where you assume, since pupils are selected by chance, 
that variables are randomly distributed. 

DeVault (1966) and Romberg and DeVault (1967) emphasize our need for realistic 
research that takes into account the complexity of the classroom setting. On 
the other hand, if you have a grandiose design that tries to take into 
account many, many factors, the study will become very complicated. Remember 
that there is a place to look at small (but not trivial) pieces (Van Engen, 
1967). 

Select or develop appropriate measuring instruments . Remember especially that 
"global" or standardized tests are not always appropriate. For example, if 
you’re testing the effect of introducing multiplication in two ways, you’ll 
find that a "global" test has a limited number of items which measure multipli- 
cation achievement. The study may result in no significant differences when 
in fact differences were present — but unmeasured. Instead of a "global" test, 
a test to measure achievement in multiplication must be constructed. 

If two different treatments are to be evaluated, the test must be carefully 
constructed so it doesn’t introduce a bias. Some research has been done in 
which the test contained a large number of items which only the experimental 
group would be able to answer (e.g., questions related to a story used to 
introduce the experimental treatment) . Thus the findings of the research 
favor the experimental group — but not because the pupils did significantly 
better on the factor being studied. 
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After you ve carefully outlined your research procedures, consider: could I 

replicate this study, that is, do it over again and expect to get the same re- 
sults? If you can’t answer "yes," replan!' Then check your plans with someone 
” ^ , ?^search get professional assistance from your research department 

or from a university or college, whenever this is possible. This step often 
makes the difference between good research and a meaningless collection of 
data, between an answer to your question and no answer. This is the time to 
clarify questions like "What data should be collected?" and "How will the data 

£_rocedures ^ contemplated should not be contemplated 
i ndependently consideration of the way in which data are to be collected 
^ analy^. People have been known to collect data and then wander around 
trying to find a statistic to use. They don’t always find one. In fact, one 
may not even exist! 

Research is improved by being tried out first of all with a pilot study- 
problems are resolved before they affect your major study. For instance, in 
oing a survey, the questionnaire should be given to a small group before it 
IS used in the study. A test should be administered to a small group, prefer- 
ably one much like the group who will be involved in the research. You want to 
be sure each is valid and reliable, that is, that each measures what it’s sup- 
posed to measure, consistently. 

It is wise to consider the timing of research. Usually it’s unwise to plan to 

begin a study on the first day of school. Beware of other things competing 

^ vacation or other projects which claim priority. Length of time 
should be appropriate to your problem— remember that most studies can’t be 
done in one day. Also remember that the longer the study, the more problems 
you may have and the more difficult control becomes. 

If you have only two classes at a grade level, the temptation is to have one 
teacher teach one treatment and the other teach the second treatment. Better 
yet, have both teachers try both— teaching some pupils by one method and some 
by the other. This eliminates some confounding, since data for each method 
can be pooled. The teachers must be doubly careful to not let biases inter- 
fere— they must do an honest job with each, despite a special preference for 

one. The way in which a teacher carries out the research plan is one of the 
most important factors. 

Be sure your s ample is appropriate for the population to which you want to 
generalize your results. There are times when it is reasonable to exclude 
data for a few children who are very different from the rest of the group 
since they may bias the research. Better yet, analyze the data for them ’ 
separately or differentially. 

Whenever appropriate to the design, pupils should he ^domly selected and 
assigned to a treatment. "How many children are needed?" cannot be answered 
in general: there’s a number that will give each study sufficient "power." 

Remember that it may be wasteful of pupil time to use samples larger than 
necessary. On the other hand, too small a number raises questions about how 
representative they are, and how far the findings can be generalized. 

There are instances in which it is feasible to conduct research only with 
intact classes. This situation presents certain problems of research design 
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which need to be considered. Campbell and Stanley (1963) provide some help on 
this type of situation. 

There is a time to pretest — ^when you think that pupils have some knowledge of 
the subject matter. But in other cases, when you can assume that pupils have 
no knowledge or equivalent knowledge (e.g., when non-decimal bases are intro- 
duced in grade 1), a pretest is not necessary. A pilot study using a pretest 
will indicate whether or not a pretest is necessary in the final study. 

It is desirable for teachers involved to keep logs of what was done day by day, 
as well as anecdotal records of particular incidents and reactions. Then 
departures from the planned procedures can be noted j these are sometimes use- 
ful in interpreting findings. Also, there is a need for somebody to keep a 
finger on things as the study progresses, to make sure procedures are being 
followed. 

Reporting and disseminating information about research should be carefully 
done, ^ important that o thers know what you have done and found . Accuracy 
in reporting is essential, as well as readability. As Weaver (1967) noted, 

"We can go a long way toward extending the impact of research if each investi- 
gator accepts the obligation to report all significant aspects of this work as 
fully as is necessary to establish the integrity of his research and of the 
conclusions drawn.” The interpretations must be derived from the data — and 
remember that there is a difference between findings and implications. 

We have frequently cited differences which are "significant” or "statistically 
significant." By this we mean that there is a specified likelihood that such 
differences would not have occurred by chance. Usually, the level of signifi- 
cance is set at .05, or .01, or .001 — thus the results might occur by chance 
only 5 times in 100, or only 1 time in 100, or only 1 time in 1000. "No sig- 
nificant differences" means that a specified level of significance was not 
reached — thus the results could occur more frequently by chance. Researchers 
Sv°t a level which seems appropriate to them in terms of the content and design 
of their study. 

In summary, as you plan how to develop and implement your research, you may 
find these questions helpful: 

(1) Is the problem practically and/or theoretically significant? 

(2) Is the problem clearly defined? 

(3) Is the design appropriate to answer the research question? 

(4) Does the design control variables? 

(5) Is the sample properly selected for the design and purpose of the 
research? 

(6) Are the measuring instruments valid and reliable? 

(7) Are the techniques of analysis of the data valid? 

(8) Are the interpretations and generalizations appropriate to the data? 

(9) Is the research adequately reported ? 
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WHAT? We look only at mathematics research in these bulletins. 

Some findings from mathematics research, especially those 
cited in Set A, might be considered in regard to other 
phases of the curriculum — and research from other areas may 
be applicable to the teaching and learning of mathematics. One caution is 
necessary: don’t take findings from one field and assume automatically that 

they are true for mathematics, or any other. It is important to recognize 
that conceptual learning is particularly important in mathematics. Mathematics 
may differ from other areas because there is a body of content (unlike language 
arts, but like science), v/hich also is sequential; therefore there may be dif- 
ferent problems for mathematics than for other areas. 

Schools probably do not need to do research on those things on which there is 
"sound" research evidence already (for instance, on the benefits to be derived 
from meaningful instruction). There are large variations, however — what may 
be true in general for large groups may not be true for particular, unique 
groups. What may be true for one topic may not be true for another. 

Teachers should test research findings in their own classrooms (Riedesel, 

1968). Remember that just because research says that something was best for a 
g^Q^P of teachers in a variety of classrooms, doesn't necessarily mean that it 
would be best for you as an individual teacher in your particular classroom. 

For instance, we're beginning to get evidence that there is an interaction be- 
tween teaching and method. Thus research may show that an inductive approach 
is "good," — yet some teachers may not be comfortable with it or can't manage 
it. An expository approach may be better for those teachers. Teachers have 
individual differences as well as pupils I In this same way, remember that 
learning modes of pupils differ, and that not all content lends itself to use 
of inductive strategies. 

Teachers must be careful not to let prior judgments influence their willing- 
ness to try out and explore: open-mindedness is important in research. Be 

willing to investigate. But being open-minded doesn't mean you don't have 
beliefs about things — just that you don't let beliefs bias the conduct of 
research. 

Often research may be generated by informal exploration that teachers make, 
which in itself is not research. Do this — but don't call it research; use it 
to generate hypotheses which can then be tested with research. 



AND THEN . . . Research is not an end in itself — it should lead to 

some kind of action. You decide to change, or not to 
change; you will accept something, you will reject 
something. It may lead to other research. ^ some- 
thing as a result of research: incorporate the conclusions of rese^ch into 

your daily teaching. 

Non-significant differences can be as important as significant differences — 
don't be disappointed or think automatically that research has "failed" when 
no significant differences result. There might in fact ^ no differences — and 
the decision is up to youl 
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®^''® °"^y ® glimpse of some of the 

p”f^ ‘■° =™sidered as schools conduct researchT^ou may wish 

o urt er into the design and implementation of research as you plan for 
your own investigations. ^ 

Good luck I 
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